Quantitative 3d reconstruction from scanning electron microscope images based on affine camera models

Scanning electron microscopes (SEMs) are versatile imaging devices for the micro-and nanoscale that find application in various disciplines such as the characterization of biological, mineral or mechanical specimen. Even though the specimen’s two-dimensional (2D) properties are provided by the acquired images, detailed morphological characterizations require knowledge about the three-dimensional (3D) surface structure. To overcome this limitation, a reconstruction routine is presented that allows the quantitative depth reconstruction from SEM image sequences. Based on the SEM’s imaging properties that can be well described by an affine camera, the proposed algorithms rely on the use of affine epipolar geometry, self-calibration via factorization and triangulation from dense correspondences. To yield the highest robustness and accuracy, different sub-models of the affine camera are applied to the SEM images and the obtained results are directly compared to confocal laser scanning microscope (CLSM) measurements to identify the ideal parametrization and underlying algorithms. To solve the rectification problem for stereo-pair images of an affine camera so that dense matching algorithms can be applied, existing approaches are adapted and extended to further enhance the yielded results. The evaluations of this study allow to specify the applicability of the affine camera models to SEM images and what accuracies can be expected for reconstruction routines based on self-calibration and dense matching algorithms. © MDPI AG. All rights reserved

Investigation of harmful slip in a rolling bearing using a thermographic camera with a derotator

Due to their high availability and versatility, rolling bearings are a standard solution for mounting and support of rotating components. The service life of an entire rotating machine is often limited by the service life of rolling bearings. This can be shorter than expected if the rolling bearing is operated in harmful operating conditions, e.g. in the presence of slip. Slip means that there is a deviation between the theoretical angular velocity of the rolling element set and the actual angular velocity. In this context, slip is harmful if it leads to increased friction and thus heating of the bearing. The occurrence and properties of slip are not completely understood yet. Therefore, it is of interest to investigate a relationship between slip and heating of the rolling bearing in order to better understand damages of the rolling bearing. In this work, a method is presented in which, in addition to slip measurements with a high-speed camera, a thermal imaging camera is used to investigate the heating of the bearing during operation. Since the rotational movement and exposure time of the camera would cause motion blur, the thermal imaging camera is operated together with a derotator to optically eliminate the rotational movement. The investigations of the rolling bearing are carried out under different operating conditions (different loads and rotational speeds), which have an influence on the slip behavior of the bearing. Thus, the potential of this investigation method for deepening the understanding of heating and friction in rolling bearings is demonstrated

Topography Measurement for Monitoring Manufacturing Processes in Harsh Conditions

High precision manufacturing, e.g. milling and grinding, which have manufacturing tolerances in the range of <10 μm require microscopic measurement techniques for the inspection of the manufactured components. These measurement techniques are very sensitive to cooling liquids and lubricants which are essential for many manufacturing processes. Therefore, the measurement of the components is usually conducted in separate and clean laboratories and not directly in the manufacturing machine. This approach has some major drawbacks, e.g. high time consumption and no possibility for online process monitoring. In this article, a novel concept for the integration of high precision optical topography measurement systems into the manufacturing machine is introduced and compared to other concepts. The introduced concept uses a reservoir with cooling liquid in which the measurement object is immersed during the measurement. Thereby, measurement disturbance by splashing cooling liquids and lubricants can effectively be avoided.BMBF/03V047

Position Determination of a Robot End-Effector Using a 6D-Measurement System Based on the Two-View Vision

A mechatronic system based on the micro-macro-kinematic consists of an industrial robot and a piezoelectric stage mounted on the robot’s end-effector and has to carry out operations like micro-assembly or micro-milling. The piezoelectric stage has to compensate the positioning error of the robot. Therefore, the position of the robot’s end-effector has to be measured with high accuracy. This paper presents a high accuracy 6D-measurement system, which is used to determine the position and orientation of the robot’s end-effector. We start with the description of the operational concept and components of the measurement system. Then we look at image processing methods, camera calibration and reconstruction methods and choose the most accurate ones. We apply the well-known pin-hole camera model to calibrate single cameras. Then we apply the epipolar geometry to describe the relationship between two cameras and calibrate them as a stereo vision system. A distortion model is also applied to enhance the accuracy of the system. The measurement results are presented in the end of the paper

Fiber-optic fringe projection with crosstalk reduction by adaptive pattern masking

To enable in-process inspection of industrial manufacturing processes, measuring devices need to fulfill time and space constraints, while also being robust to environmental conditions, such as high temperatures and electromagnetic fields. A new fringe projection profilometry system is being developed, which is capable of performing the inspection of filigree tool geometries, e.g. gearing elements with tip radii of 0.2 mm, inside forming machines of the sheet-bulk metal forming process. Compact gradient-index rod lenses with a diameter of 2 mm allow for a compact design of the sensor head, which is connected to a base unit via flexible high-resolution image fibers with a diameter of 1.7 mm. The base unit houses a flexible DMD based LED projector optimized for fiber coupling and a CMOS camera sensor. The system is capable of capturing up to 150 gray-scale patterns per second as well as high dynamic range images from multiple exposures. Owing to fiber crosstalk and light leakage in the image fiber, signal quality suffers especially when capturing 3-D data of technical surfaces with highly varying reflectance or surface angles. An algorithm is presented, which adaptively masks parts of the pattern to reduce these effects via multiple exposures. The masks for valid surface areas are automatically defined according to different parameters from an initial capture, such as intensity and surface gradient. In a second step, the masks are re-projected to projector coordinates using the mathematical model of the system. This approach is capable of reducing both inter-pixel crosstalk and inter-object reflections on concave objects while maintaining measurement durations of less than 5 s.DFG/CRC/TR 7

Texture direction analysis of micro-topographies using fractal geometry

In recent years fractal geometry has been repeatedly shown to help describe and characterize micro-topographies. Important properties of micro-topographies include roughness or texture direction. Consequently, in this work, fractal geometry is investigated to determine texture direction. For this purpose, synthetic data and real height maps of different micro-topographies are evaluated using the fractal power spectrum density method. The results are compared with a manual determination of the texture direction and determining the texture direction according to ISO 25 178 using the S td parameter. The results show that the fractal method is more accurate than the currently standardized method. Another advantage is that secondary texture directions can be detected. Thus, the fractal method is well suited for characterizing micro-topographies and can complement existing parameters from ISO 25 178

Development of a compact low coherence interferometer based on GPGPU for fast microscopic surface measurement on turbine blades

Vertical scanning interferometry (VSI) techniques are widely used to profile microscopic surface structures of industrial products. This paper introduces a high-precision fast optical measurement system with an optimized small sensor head for the measurement of precision surfaces on a turbine blade or blisks (blade integrated discs). The non-contact measurement system is based on a low coherence interferometer (LCI), which is capable of fast profiling of 3D sample surface with a nanometer resolution and has a larger measurement range compared to conventional microscopes. This results in a large amount of sampled data and a high computational time for the evaluation of the data. For this reason, the used evaluation algorithm in this paper is accelerated by the Compute Unified Device Architecture (CUDA) technology, which allows parallel evaluation of the data stack on independent cores of a General Purpose Graphics Processing Unit (GPGPU). As a result, the GPU-based optimized algorithm is compared with the original CPU-based single-threaded algorithm to show the approximate 60x speedup of computing the Hilbert Transformation, which is used to find the depth position in the correlogram of each pixel of the sampled data. The main advantage of the GPU computing for the evaluation algorithm of the LCI is that it can reduce the time-consuming data evaluation process and further accelerates the whole measurement. © 2015 SPIE.DFG/SFB/87

Comparison of LASER and LED illumination for fiber optic fringe projection

The inspection of functional elements is a crucial part of modern production cycles. However, with higher integration of production machinery and products, the accessibility for measurement systems is more and more limited. A solution for this problem can be found in endoscopy techniques, which are able to transport the image information for optical measurement methods. In this paper, an optical inspection system based on the fringe projection proifllometry technique is presented. The iflber-optic fringe projection system uses two high-resolution image iflbers to connect a compact sensor head to the pattern generation and camera unit. In order to keep inspection times low, the system is developed with particular focus on fast projection times. This can be achieved by using a digital micro-mirror device, which is capable of projecting grey-scale patterns at a rate of more than 10 images per second. However, due to the low numerical aperture of the optical iflbers, a limiting factor for the pattern rate is the illumination path of the pattern generator. Two different designs of the illumination path are presented, which are based on a LASER light source as well as a LED light source. Due to low beam divergence and high intensities LASERs are well suited for iflber coupling. Unfortunately, the coherent property of the light has negative effects in certain measurement applications, as interference patterns, the so called speckle, appear on rough surfaces. Although speckle reducing methods are employed in the LASER beam path, the emergence of interference cannot be prevented completely. As an alternative, an illumination path based on a LED light source is demonstrated. To compare the effects of the speckle, based on measurements on a planar calibration standard both designs are compared in terms of phase noise, which is directly related to the noise in the reconstructed 3-D point data. Additionally, optical power measurements of both methods are compared to give an estimation of coupling eiflciency. Finally, the capabilities of the system are shown based on measurements of a micro-contour standard. © 2016 SPIE.DFG/CRC/TR 7

Fast in-situ tool inspection based on inverse fringe projection and compact sensor heads

Inspection of machine elements is an important task in production processes in order to ensure the quality of produced parts and to gather feedback for the continuous improvement process. A new measuring system is presented, which is capable of performing the inspection of critical tool geometries, such as gearing elements, inside the forming machine. To meet the constraints on sensor head size and inspection time imposed by the limited space inside the machine and the cycle time of the process, the measuring device employs a combination of endoscopy techniques with the fringe projection principle. Compact gradient index lenses enable a compact design of the sensor head, which is connected to a CMOS camera and a flexible micro-mirror based projector via flexible fiber bundles. Using common fringe projection patterns, the system achieves measuring times of less than five seconds. To further reduce the time required for inspection, the generation of inverse fringe projection patterns has been implemented for the system. Inverse fringe projection speeds up the inspection process by employing object-adapted patterns, which enable the detection of geometry deviations in a single image. Two different approaches to generate object adapted patterns are presented. The first approach uses a reference measurement of a manufactured tool master to generate the inverse pattern. The second approach is based on a virtual master geometry in the form of a CAD file and a ray-tracing model of the measuring system. Virtual modeling of the measuring device and inspection setup allows for geometric tolerancing for free-form surfaces by the tool designer in the CAD-file. A new approach is presented, which uses virtual tolerance specifications and additional simulation steps to enable fast checking of metric tolerances. Following the description of the pattern generation process, the image processing steps required for inspection are demonstrated on captures of gearing geometries. © 2016 SPIE.DFG/CRC/TR 7

Endoscopic fringe projection for in-situ inspection of a sheet-bulk metal forming process

Sheet-bulk metal forming is a new production process capable of performing deep-drawing and massive forming steps in a single operation. However, due to the high forming forces of the forming process, continuous process control is required in order to detect wear on the forming tool before production quality is impacted. To be able to measure the geometry of the forming tool in the limited space of forming presses, a new inspection system is being developed within the SFB/TR 73 collaborative research center. In addition to the limited space, the process restricts the amount of time available for inspection. Existing areal optical measurement systems suffer from shadowing when measuring the tool's inner elements, as they cannot be placed in the limited space next to the tool, while tactile measurement systems cannot meet the time restrictions for measuring the areal geometries. The new inspection system uses the fringe projection optical measurement principle to capture areal geometry data from relevant parts of the forming tool in short time. Highresolution image fibers are used to connect the system's compact sensor head to a base unit containing both camera and projector of the fringe projection system, which can be positioned outside of the moving parts of the press. To enable short measurement times, a high intensity laser source is used in the projector in combination with a digital micro-mirror device. Gradient index lenses are featured in the sensor head to allow for a very compact design that can be used in the narrow space above the forming tool inside the press. The sensor head is attached to an extended arm, which also guides the image fibers to the base unit. A rotation stage offers the possibility to capture measurements of different functional elements on the circular forming tool by changing the orientation of the sensor head next to the forming tool. During operation of the press, the arm can be travelled out of the moving parts of the forming press. To further reduce the measurement times of the fringe projection system, the inverse fringe projection principle has been adapted to the system to detect geometry deviations in a single camera image. Challenges arise from vibrations of both the forming machine and the positioning stages, which are transferred via the extended arm to the sensor head. Vibrations interfere with the analysis algorithms of both encoded and inverse fringe projection and thus impair measurement accuracy. To evaluate the impact of vibrations on the endoscopic system, results of measurements of simple geometries under the influence of vibrations are discussed. The effect of vibrations is imitated by displacing the measurement specimen during the measurement with a linear positioning stage. The concept of the new inspection system is presented within the scope of the TR 73 demonstrational sheet-bulk metal forming process. Finally, the capabilities of the endoscopic fringe projection system are shown by measurements of gearing structures on a forming tool compared to a CAD-reference. © 2015 SPIE.DFG/CRC/TR 7